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KNOWLEDGE -BASED STRATEGIES APPLIED TO N-BEST LISTS IN 
AUTOMATIC SPEECH RECOGNITION SYSTEMS 

BACKGROUND OF THE INVENTION 

5 Technical Field 

The present invention relates generally to automatic 
speech recognition (ASR) and, more particularly, to 
recognition of spoken alphabet and alpha-numeric strings 
using knowledge -based strategies applied to a list of 
10 hypothesized recognition results. 
Description of the Related Art 

ASR is used for various recognition tasks, including 
recognizing digit strings spoken by telephone callers. 
These digit strings typically represent credit card 

15 numbers, telephone numbers, bank account numbers, social 
security numbers and personal identification numbers (PIN) . 

Speech recognition is an imperfect art. Achieving 
high accuracy is difficult because multiple variables 
typically exist including, e.g., differences in 

20 microphones, speech accents, and speaker abilities. 
Recognizing spoken digit strings is particularly difficult 
because individual digits are short in duration, have a 
high degree of inter-digit acoustic confusibility, and are 
often co-articulated with adjacent digits. When digit - 

25 string (and alphabet or alpha-numeric) recognition is 
performed over a telephone network, the task is even more 
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difficult, owing to the noise and bandwidth limitations 
imposed on the speech signal. Recognizing a string of 
spoken digits correctly requires that each digit be 
recognized accurately. Recognizing strings of spoken 
digits at high accuracy requires per-digit accuracies that 
are extremely high - in excess of 99%. The state of the 
art over- the -telephone digit recognition attempts to 
achieve about a 98% per-digit accuracy. Alphanumeric 
recognition over-the-telephone is even more difficult, with 
state-of-the-art recognition accuracy around 75% per 
character. 

There is thus a need for a more accurate digit 
recognition technique, particularly for recognizing spoken 
digit strings over a telephone network. 
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BRIEF SUMMARY OF THE INVENTION 

A primary object of the invention is to provide a 
method and apparatus for high accuracy recognition of 
spoken digit strings. 
5 A more particular object of the invention is to 

provide new techniques for recognizing spoken digit 
strings, preferably using knowledge -based strategies 
applied to a list of hypothesized digit strings. 

It is still another more general object of this 
10 invention to implement various knowledge -based strategies 
for controlling a speech recognizer. 

These and other objectives are accomplished by a 
method and system for recognizing spoken digit strings. In 
accordance with a preferred embodiment of the invention, a 
15 spoken digit string is analyzed by a speech recognizer, 
which generates a list of hypothesized digit strings 
arranged in ranked order based on a likelihood of matching 
the spoken digit string (referred to herein as the "N-best 
list") . The individual hypothesized strings are then 
20 analyzed to determine whether they satisfy a given 
constraint, beginning with the hypothesized string having 
the greatest likelihood of matching the spoken string. The 
first hypothesized string in the list satisfying the 
constraint is selected as the recognized string. 
25 Various types of constraints may be used to validate 
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the hypothesized digit strings including, e.g., checksum 
constraints, valid data string matching constraints, and 
the like. 

In accordance with further embodiments of the present 
invention, if none of the hypothesized digit strings in the 
N-best list satisfies the specified constraint, alternative 
verification techniques can be applied to determine the 
correct digit string. 

The foregoing has outlined some of the more pertinent 
objects and features of the present invention. These 
objects should be construed to be merely illustrative of 
some of the more prominent features and applications of the 
invention. Many other beneficial results can be attained 
by applying the disclosed invention in a different manner 
or modifying the invention as will be described. 
Accordingly, other objects and a fuller understanding of 
the invention may be had by referring to the following 
Detailed Description of the Preferred Embodiment. 
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BRIEF DESCRIPTION OF THE DRAWINGS 

For a more complete understanding of the present 
invention and the advantages thereof, reference should be 
made to the following Detailed Description taken in 
connection with the accompanying drawing in which: 

FIGURE 1 is a flowchart illustrating a technique for 

recognizing a spoken digit string in accordance with the 
invention. 
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DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT 

As discussed above, the present invention is directed 
to a robust method and system for accurately recognizing 
spoken digit strings. According to the present invention, 
5 the inventive technique may be used within or as an adjunct 
to a known digit recognizer or recognition engine. The 
digit recognizer or recognition engine receives a spoken 
input string and generates multiple recognition hypotheses 
for each spoken digit string. This is a known function 

10 that is available from several prior art systems (namely, 
recognition systems, applications and the like) including, 
without limitation, the Vpro/Continuous speech recognition 
engine, the VR/Continuous Speech Recognition engine, and 
the SpeechWave Standard speech recognition product, all 

15 currently developed and marketed by VCSI. In general, any 
speech recognition engine that employs a Viterbi beam- 
search technique can be configured to supply multiple- 
hypotheses in this manner. Other techniques for supplying 
multiple digit string hypotheses are also well known in 

20 prior art. As is well-known, the hypothesized digit strings 
are arranged in a rank-ordered fashion based on a 
likelihood of matching the spoken digit string (the N-best 
list) . In accordance with the invention, this multi-choice 
feature is used in conjunction with various knowledge based 

25 recognition strategies to accurately recognize the spoken 
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digit string. 

Briefly, the inventive technique preferably analyzes 
the recognizees first choice digit string (i.e., the first 
item in the probability sorted N-best list) to determine 
5 whether the first choice satisfies a given knowledge-based 
recognition constraint. If the constraint is satisfied, 
then that digit string is validated, i.e., it is declared 
to be the correct number. If the first choice does not 
satisfy the constraint, the recognizer 1 s second choice is 
10 considered, and so forth, until a valid digit string is 
found . 

If none of the hypothesized digit strings meet the 
constraint, then a rejection is declared, and the caller 
may be asked to repeat the digit string for a new analysis. 

15 Alternately, as will be described below, additional (or 
supplemental) verification techniques are used to determine 
the correct digit string. 

Figure 1 generally illustrates the inventive 
recognition process 10. First, at step 12, a user (who may 

20 be a telephone caller) is prompted to provide a spoken 
digit string such as, e.g., a credit card number. The 
system receives the spoken digit string at 14. The digit 
recognizer then analyzes the spoken digit string at step 16 
and generates a rank ordered list of hypothesized digit 
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strings (the N-best list) based on the level of confidence 
it has in recognizing the spoken string. The hypothesized 
digit strings in the list are arranged in ranked order from 
the most likely to the least likely correct match to the 
5 spoken string. As discussed above, this is a known 
functionality. Then, at step 18, the first hypothesized 

string in the list is analyzed. If the string satisfies a 
given constraint at step 20, then the hypothesized string 

is validated at 22 (i.e., it is selected as being the 

10 correctly recognized string) . If the constraint is not 
satisfied, then a determination is made as to whether there 
are any other hypothesized strings on the list at step 24. 

If so, then the next string on the list is examined at step 
26. The process then goes to step 20 and repeats the 
15 subsequent steps until the constraint is satisfied. If 
none of the hypothesized strings on the list satisfy the 
constraint, then the recognition process will then be 
deemed to have been unsuccessful at 28, and the process can 

optionally return to step 12 to ask the telephone caller to 
20 repeat the spoken digit string. Alternately, after step 
28, other verification techniques (described below) can be 
applied to determine the correct digit string. 

According to one feature of the present invention, 
various types of knowledge-based strategies are applied to 
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the N-best list to validate hypothesized digit strings. 
Checksums 

For example, one knowledge -based strategy is a 
checksum approach. Using a checksum strategy, each 
5 hypothesized digit strings in the sorted N-best list is 
analyzed until a hypothesized digit string that correctly 
checksums is found. This hypothesized digit string is then 
verified as the answer. 

As is known, checksum schemes are frequently used with 

10 various kinds of numeric data including, e.g., credit-card 
numbers, bank-account numbers, and other kinds of account 
numbers. For purposes of illustration, a credit card 
number is used as an example of a recognition task where 
the checksum strategy is applied. 

15 In general, credit card numbers are comprised of a 

fixed number of digits, typically fifteen or sixteen. The 
last digit of the credit card number is referred to as the 
checksum digit. The checksum digit represents a 

mathematical combination of the other digits in the credit 

20 number. Various known checksum algorithms can be utilized. 

Another checksum algorithm known as the Luhn checksum 
algorithm is commonly used for credit card numbers. The 
Luhn checksum is calculated as follows: For a card with an 
even number of digits, every odd numbered digit is doubled, 

25 and nine is subtracted from the product if the product is 
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greater than 9. The even digits as well as the doubled-odd 
digits are then added* The result must be a multiple of 10 
or the number is not a valid card number and is rejected. 
If the card has an odd number of digits, the same addition 
5 is performed, but with the doubling of the even numbered 
digits instead. 

Using a checksum strategy together with the N-best 
list to validate a credit card number dramatically improves 
recognition accuracy. For example, recognizing credit card 

10 numbers without using checksum information will yield 
string accuracies of about 75% under typical conditions. 
Under the same conditions, recognizing credit card numbers 
using the N-best list and the checksum information yields 
accuracies of about 95%. Moreover, the "false acceptance" 

15 rate for this task (i.e., where the recognizer returns a 
checksum conforming result that is incorrect) is extremely 
low, usually less than 1%. The remaining errors (around 4% 
of the total) are rejections, requiring the application to 
reprompt or fallback to human intervention. For most 

20 applications, rejection errors are preferable to false 
acceptances . 
Database Match 

Another knowledge based strategy is matching to a 
database. Many applications of digit-string recognition 

25 (e.g., postal codes, license plates, catalog sales, 
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electric-utility account information systems) have access 
to databases that list the valid entries. Accordingly, the 
N-best list can be screened in a very similar fashion to 
using the previously described checksum strategy, except 
5 that the acceptance criterion preferably is now an exact 
match to an item in the database. Because many of these 
databases are constructed in order to reduce the likelihood 
that typing errors will cause the wrong account to be 
accessed, this knowledge based strategy is a very powerful 

10 tool for processing the N-best list. Moreover, the 
database strategy is also useful for alphabet strings. 

While it is possible to "precompile" such databases 
into a "grammar" - and therefore apply the database 
constraints before the recognition - this is often 

15 impractical because the databases change frequently, making 
continual recompilation necessary. Also, when the 

databases are large, grammar-recompilation can be very time 
consuming. As such, verifying the N-best list against the 
database, for example, by using fast matching techniques 

20 known in the prior art, is often the only practical way to 
apply such constraints. 

Another application of the database-match technique is 
particularly suited for use with PIN numbers, e.g., in 
voice-controlled voicemail systems or voice-controlled 

25 banking applications. In these applications, it is known 
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that the user inputs both an account number and a PIN 
number as a security measure. The following are examples 
of two ways of using the N-best screening technique here. 

First, the technique is applied only to the PIN 
5 number. Assuming the account number is correct, there is 
usually some database lookup of the account number where 
the PIN number can be accessed. In such case, each item of 
the N-best list may be checked to see if it matches the PIN 
number to approve the entry. The ASR part of the 

10 application need not "know" explicitly what the PIN number 
is. All that is required is a string-match at some point 
in the N-best screening process. The actual PIN number can 
be discarded to preserve security. 

Second, the technique may be applied to the account 

15 number and PIN number concurrently. In this case, the N- 
best lists for both the account number and PIN number 
recognitions are kept. Each account number hypothesis is 
looked up in the database to access the associated PIN 
number. If there is no match on the account number (or no 

20 "fuzzy" match, a technique described below) , then this 
account number is rejected. If there is an account number 
match, then a subsequent match is performed on the PIN 
number against the N-best list for that utterance. This 
process can be repeated until the best possible combined 

25 match of account number and PIN number is achieved. 
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An advantage of the database verification technique 
(for the N-best list) is that it can be applied to alphabet 
string recognition and alphanumeric string recognition as 
well as pure digit string recognition. (Checksum 
5 verification can also be applied this way by assigning a 
numerical value to the alphabet characters J 
Digit Positional Constraints 

If there are positional constraints on digits (or 
alpha characters) , the answers in the N-best list can be 

10 checked to verify that these constraints are met. Answers 
that do not obey these constraints may be rejected. While 
it is possible to apply these constraints before the 
recognition, for example, by using digit "micro-grammars, " 
this is sometimes not practicable. In these circumstances, 

15 these grammar constraints can be beneficially applied to 
the N-best list. 
Digit String Length Constraints 

Similarly, digit-string (or alphabetic- string or 
alphanumeric- string) length constraints can be applied to 

20 the N-best list scheme. Again, it is possible to apply 
these constraints at recognition time, but sometimes this 
information is not available or needs to be hidden for 
security reasons, e.g., when verifying PIN numbers. In 
these cases the N-best list can be screened for items 

25 conforming to known length constraints. 
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Supplemental Techniques 

In certain circumstances, the knowledge based 
recognition strategy does not generate a match to one of 
the entries of the N-best list. In such case, it may be 
5 desirable to supplement the knowledge-based strategy. The 
present invention also contemplates the use of such 
supplemental techniques if necessary. 

Thus, for example, assume that none of the N-best list 
choices match any entries in the database being searched 

10 (in the exact database matching technique described above) . 
In that event, a supplemental technique, such as a "fuzzy" 
matching scheme, is applied. As is well-known, this 
technique does not require an exact database match. 
Instead, each answer of the N-best list is compared in a 

15 "fuzzy" manner to the database of valid numbers (or 
alphabetic or alphanumeric strings) . 

The fuzzy matching criterion may be any of a number of 
standard techniques, mostly involving well-known dynamic- 
programming algorithms. For example, the Levenshtein 

2 0 distance algorithm (see: Sankoff, D. and Joseph B. Kruskal, 
"Time Warps, String Edits and Macromolecules : The Theory 
and Practice of Sequence Comparison," pp. 18-21, Addison- 
Wesley, 1983) may be applied. In this algorithm, one 
string is "matched" against another by determining the 

25 sequence of substitutions, deletions, and insertions 
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required to "transform" one string into the other. The 
"distance" between the two strings is the minimum number of 
such "corrections" (substitutions + deletions + insertions) 
required to perform the transformation. 
5 A "weighted" version of the Levenshtein algorithm may 

also be applied, in which certain corrections are deemed to 
"cost" more than others. For example, when performing 
digit recognition in noisy conditions, it is common for the 
ASR algorithm to "insert" some digits - such as "oh" and 

10 "eight" - as hypotheses. A weighted matching algorithm may 
therefore decide to penalize such insertions less than 
other classes of insertions, and/or substitutions and 
deletions. In this way, the particular limitations of the 
ASR technology can be accounted for in order to achieve 

15 more robust database-matching. 

This approach also applies to alphabetic and 
alphanumeric recognition. For alphabet recognition the 
"weighted" matching criterion can be very useful as there 
are certain sets of characters that are often very 

20 confusing to ASR systems. For example, it is difficult for 
state-of-the-art ASR algorithms to distinguish among the 
"E-set" (b, c, d, e, g, p, t, v) of alphabet characters, 
especially in band- limited conditions such as those 
typically found in telephone networks (both fixed and 

25 wireless) . In this case, the Levenshtein distance can be 
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modified to penalize substitution modes among these 

characters less than the other error modes. 

Variants 

In accordance with a further embodiment of the 
5 invention, the N-best results from two recognition attempts 
can be intelligently combined to ascertain the actual 
spoken string. With this "2 -utterance" or "repeated 
utterance" technique, the following procedure preferably is 
followed: 

10 a) The user is prompted for a number (or alphanumeric 

string) once, 

b) The recognition is run on the spoken utterance 
using the digit recognizer, and the N-best list is obtained 
("LIST 1") . 

15 c) Next, the digit recognizer's "confidence" measure 

is used to approve or reject the top answer in the N-best 
list. If the confidence level is sufficiently high, the 
utterance is accepted. However, if the confidence- level is 
below a given threshold, the user is prompted to repeat the 

2 0 string. 

d) The recognition is then run on the repeated 
utterance, and another N-best list is obtained ("LIST 2") . 

e) Next, LIST 1 is used as a "database" in order to 
verify one of the hypotheses in LIST 2 using the database 

25 matching or fuzzy matching approaches described above. In 
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effect, the first hypothesis in LIST 2 that also occurs in 
LIST 1 is selected. Alternately, the checking order can be 
reversed, i.e., the first hypothesis in LIST 1 that also 
occurs in LIST 2 can be selected. If using the fuzzy 
5 technique, the item in LIST 2 that best matches a 
hypothesis in LIST 1 (or visa versa) is selected. 

In accordance with yet another embodiment of the 
invention, if none of the hypothesized digit strings are 
found to satisfy a specified constraint (e.g., checksum, 

10 database match, etc.), then a further verification can be 
applied. With this technique, the N-best list is used as a 
means to generate other hypotheses, which are then analyzed 
to determine if they satisfy the given constraint. For 
example, suppose you N-best list contains the following 

15 three hypotheses: 

(1) 1 2 3 4 5 

(2) 4 2 3 4 5 

(3) 12 3 15 

Then, by combining information from these three choices, it 
2 0 is reasonable to hypothesize the string "4 2 3 1 5" as an 
alternative. Even though "42315" does not appear in 
the N-best list, it can be "synthesized" by observing the 
4->l "close call" in the fourth position of hypotheses (1) 

and (3) , and the l->4 close call in the first position of 

25 hypotheses (1) and (2) . (All of the other permutations of 
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This generated string can be checksumed or otherwise 
analyzed to determine if it satisfies the specified 
constraint . 

5 The hypothesis-generation technique can also be 

applied to the repeated utterance technique described above 
by combining the N-best lists from both recognitions (i.e., 
LIST 1 and LIST 2) into a single N-best list* Then, the 
hypothesis -generation technique is applied. The combined 

10 lists provide richer possibilities for permutations. 

The digit recognition algorithms in accordance with 
the invention preferably comprise software, and thus one of 
the preferred implementations of the invention is as a set 
of instructions (program code) in a code module resident in 

15 the random access memory of a general purpose computer. 
Until required by the computer, the set of instructions may 
be stored in another computer memory, e.g., in a hard disk 
drive or in a removable memory such as an optical disk (for 
eventual use in a CD ROM) or a floppy disk (for eventual 

20 use in a floppy disk drive) , or downloaded via the Internet 
or some other computer network. In addition, although the 
various methods described are conveniently implemented in a 
computer selectively activated or reconfigured by software, 
one of ordinary skill in the art would also recognize that 

25 such methods may be carried out in hardware, in firmware, 
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or in more specialized apparatus or devices constructed to 
perform the required method steps . 

A representative computer on which the inventive 
operation is performed has a processor (e.g., Intel-, 

5 PowerPC®- or RISC®-based) , random access or other volatile 
memory, disc storage, a display having a suitable display 
interface, input devices (mouse, keyboard, and the like) , 
and appropriate communications devices for interfacing the 
computer to a computer network. Random access memory 
10 supports a computer program that provides the functionality 
of the present invention. 

Having thus described our invention, what we claim as 
new and desire to secure by Letters Patent is set forth in 
the following claims. 
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CLAIMS 

1. A method of recognizing a spoken digit string, 

comprising: 

(a) receiving the spoken digit string; 
5 (b) analyzing the spoken digit string to generate a 

list of hypothesized digit strings arranged in ranked order 
based on a likelihood of matching the spoken digit string; 

(c) using a given knowledge based recognition 
strategy, determining whether individual hypothesized 

10 strings of said list satisfy a given constraint beginning 
with the string having the greatest likelihood of matching 
said spoken string; and 

(d) selecting the first string in the list satisfying 
the constraint as the recognized string. 

15 

2 . The method of Claim 1 wherein said given 

knowledge based recognition strategy comprises a database 
matching scheme. 

20 3. The method of Claim 2 wherein step (c) comprises 

searching a database of valid data strings to determine 
whether any of the hypothesized digit strings match one of 
the valid digit strings. 
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4 . The method of Claim 1 wherein the knowledge based 
recognition strategy is a checksum scheme. 

5 . The method of Claim 4 wherein the spoken digit 

5 string includes a checksum digit, and wherein step (c) 
comprises calculating a checksum of the hypothesized digit 
strings and determining whether the checksum matches the 
value of the checksum digit. 

10 6. The method of Claim 4 further comprising the 

following steps: 

if none of the hypothesized digit strings satisfies 
the constraint : 

(e) generating an additional hypothesized digit 
15 string based on information gathered from the hypothesized 

digit strings in the list; 

(f) analyzing the additional hypothesized digit 
string to determine if the checksum scheme is satisfied; 

(g) if SO; validating the additional hypothesized 
2 0 digit string. 

7. The method of Claim 4 wherein the checksum scheme 
utilizes a Luhn Checksum algorithm. 
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8. The method of Claim 1 further comprising: 

(e) if none of the hypothesized strings satisfy said 
constraint, using a supplemental matching technique to 
select the hypothesized digit string that most closely 
5 satisfies the constraint . 

9 . The method of Claim 8 wherein the supplemental 
matching technique is a fuzzy matching scheme, 

10 10. The method of Claim 9 wherein the fuzzy matching 

scheme determines the number of corrections needed to match 
each hypothesized digit string with one of a set of valid 
digit strings. 

15 11. The method of Claim 10 wherein the corrections 

comprise digit substitutions, deletions and additions. 

12 . The method of Claim 10 wherein the corrections 
are weighted. 

20 

13 . The method of Claim 1 wherein the knowledge based 

recognition strategy is a digit positional strategy and the 
constraining is a given digit position. 
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14 . The method of Claim 1 wherein the knowledge based 

recognition strategy is a digit string length strategy and 
the constraint is a given digit string length. 

15. The method of Claim 1 further comprising: 

if none of the hypothesized digit strings satisfy the 
constraint : 

(e) prompting entry of a second spoken digit string; 

(f) analyzing the second spoken digit string to 
generate a second list of hypothesized digit strings 
arranged in ranked order based on a likelihood of matching 
the second spoken digit string; 

(g) determining whether individual hypothesized 
strings of the second list match one of the hypothesized 
digit strings in the list generated in step (b) in order 
beginning with the string having the greatest likelihood of 
matching the second spoken string; and 

(h) selecting as the recognized string the first 
string in the second list matching one of the hypothesized 
digit strings generated in step (b) . 

16. The method of Claim 1 further comprising: 

if none of the hypothesized digit strings satisfy said 
constraint : 
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(e) prompting entry of a second spoken digit string; 

(f) analyzing the second spoken digit string to 
generate a second list of hypothesized digit strings 
arranged in ranked order based on a likelihood of matching 

5 the second spoken digit string; 

(g) determining whether individual hypothesized 
strings of the list generated in step (b) match one of the 
hypothesized digit strings in the second list in order 
beginning with the string having the greatest likelihood of 

10 matching the spoken string received in step (a) ; and 

(h) selecting as the recognized string the first 
string in said list generated in step (b) matching one of 
the hypothesized digit strings of said second list. 

15 17. The method of Claim 1 further comprising 

repeating the recited steps if none of the hypothesized 
strings match the constraint. 

18. The method of Claim 1 further comprising the step 
20 of prompting entry of a spoken digit string prior to step 
(a) . 
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19. A method of recognizing a spoken digit string, 
comprising: 

(a) prompting entry of a spoken digit string; 

(b) receiving the spoken digit string; 

5 (c) analyzing the spoken digit string to generate a 

list of hypothesized digit strings arranged in ranked order 
based on a likelihood of matching the spoken digit string; 

(d) using a given knowledge based recognition 
strategy, determining whether the hypothesized string of 

10 the list having the greatest likelihood of matching said 
spoken string satisfies a given constraint; 

(e) if in step (d) the constraint is found to be 
satisfied, then validating the hypothesized string, and if 
not, removing the hypothesized string from the list and 

15 repeating steps (d) and (e) for the remaining hypothesized 
strings in said list. 

20. The method of Claim 19 wherein the given 
constraint is a valid string database match. 

20 

21. The method of Claim 20 wherein step (d) comprises 

searching a database of valid data strings to determine 
whether the hypothesized digit strings match one of the 
valid digit strings . 
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22. The method of Claim 19 wherein the given 
constraint is a checksum. 

23 . The method of Claim 22 wherein the spoken digit 

string includes a checksum digit, and wherein step (d) 

calculates a checksum of the hypothesized digit strings and 

determines whether the checksum matches the value of the 
checksum digit. 

24. The method of Claim 22 further comprising: 

if none of the hypothesized digit strings satisfies 

the constraint: 

(f) generating an additional hypothesized digit 
string based on information gathered from the hypothesized 
digit strings in the list; and 

(g) analyzing the additional hypothesized digit 
string to determine if the checksum scheme is satisfied, 
and if so, validating the additional hypothesized digit 
string. 
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25. A computer program product in a computer- readable 
medium for use in a computer for recognizing a spoken digit 
s t r ing , compr i s ing : 

means for analyzing a spoken digit string to generate 
a list of hypothesized digit strings arranged in ranked 
order based on a likelihood of matching the spoken digit 
string; 

knowledge based recognition means for analyzing 
individual hypothesized strings of the list to determine 
whether they satisfy a given constraint in order beginning 
with the string having the greatest likelihood of matching 
the spoken string; and 

means for selecting the first string in said list 
satisfying said constraint as the recognized string. 



- 27 - 



005494 .00039 : 04 02907 .01 



Atty. Dkt. No. 5494:39 PATENT 

KNOWLEDGE -BASED STRATEGIES APPLIED TO N-BEST LISTS IN 
AUTOMATIC SPEECH RECOGNITION SYSTEMS 

ABSTRACT OF THE DISCLOSURE 

5 A highly accurate technique for recognizing spoken 

digit strings is described. A spoken digit string is 
received and analyzed by a speech recognizer, which 
generates a list of hypothesized digit strings arranged in 
ranked order based on a likelihood of matching the spoken 

10 digit string. The individual hypothesized strings are then 
analyzed in order beginning with the hypothesized string 
having the greatest likelihood of matching the spoken 
string to determine whether they satisfy a given 
constraint. The first hypothesized string in the list 

15 satisfying the constraint is selected as the recognized 
string. 
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DECLARATION AND POWER OF ATTORNEY FOR 



PATENT APPLICATION 

As a below named inventor, I hereby declare that: 

My residence, post office address and citizenship are as stated below next to 
my name; 

I believe I am the original, first and sole inventor (if only one name is listed 
below) or an original, first and joint inventor (if plural names are listed below) of the 
subject matter which is claimed and for which a patent is sought on the invention 
entitled: 

KNOWLEDGE-BASED STRATEGIES APPLIED TO N-BEST LISTS IN AUTOMATIC 

SPEECH RECOGNITION SYSTEMS 

the specification of which (check one): 

S is attached hereto. 

P was filed on ; 

as Application Serial No. 

and which was amended on ( (if applicable) 

I hereby state that I have reviewed and understand the contents of the above 
idantified specification, including the claims, as amended by any amendment 
referred to above. 

I acknowledge the duty to disclose information which is material to the 
patentability of this application in accordance with Title 37, Code of Federal 
Regulations, §1.56. 

I hereby claim foreign priority benefits under Title 35, United States Code, 
11 19 of any foreign application(s) for patent or inventor's certificate listed below 
and have also identified below any foreign application for patent or inventor's 
certificate having a filing date before that of the application on which priority is 
claimed: 

Prior Foreign Application(s): Priority Claimed 
(Number) (Country) (Day/Month/Year) Yes No 



I hereby claim the benefit under Title 35, United States Code, 11 20 of any 



United States applications) listed below and, insofar as the subject matter of each 
of the claims of this application is not disclosed in the prior United States 
application in the manner provided by the first paragraph of Title 35, United States 
Code, § 1 1 2, I acknowledge the duty to disclose information material to the 
patentability of this application as defined in Title 37, Code of Federal Regulations, 
§1 .56 which occurred between the filing date of the prior application and the 
national or PCT international filing date of this application: 



(Application Serial #) (Filing Date) (Status) 

I hereby declare that all statements made herein of my own knowledge are 
true and that all statements made on information and belief are believed to be true; 
and further that these statements were made with the knowledge that willful false 
statements and the like so made are punishable by fine or imprisonment, or both, 
under Section 1001 of Title 18 of the United States Code and that willful false 
statements may jeopardize the validity of the application or any patent issued 



POWER OF ATTORNEY: As a named inventor, I hereby appoint the following 
attorneys and/or agents to prosecute this application and transact all business in 
the Patent and Trademark Office connected therewith. 

David H. Judson, Reg. No. 30,467 
Douglas A. Sorensen, Reg. No. 31,570 
Andrea P. Bryant, Reg. No. 28,191. 

Send correspondence to: David H. Judson, Hughes & Luce, LLP., 1717 
Main Street, Suite 2B00, Dallas, Texas 75201 and direct all telephone calls to Mr. 
Judson at 214/939-5672. 

FULL NAME OF FIRST Thomas B. Schalk 

INVENTOR: 



thereon. 



DATE: 



INVENTOR'S SIGNATURE: 




RESIDENCE: 



6637 Muirfield Circle 
Piano, Texas 75093 
DtAajfot^ County 



CITIZENSHIP: 



U.S.A. 
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My residency post office address and citizenship are as stated below next to my name; 

I believe I am the original, first and sole inventor (if only one name is listed below) or an original, 
first and joint inventor (if plural names are listed below) of the subject matter which is claimed and for which 
a patent is sought on the invention entitled: 

KNOWLEDGE-BASED STRATEGIES APPLIED TO N-BEST LISTS IN AUTOMATIC SPEECH 

RECOGNITION SYSTEMS 

the specification of which (check one): 

S is attached hereto. 

□ was filed on ; 

as Application Serial No. 

and which was amended on (if applicable) 

I hereby state that I have reviewed and understand the contents of the above identified 
specification, including the claims, as amended by any amendment referred to above. 

I acknowledge the duty to disclose information which is material to the patentability of this 
application in accordance with Title 37, Code of Federal Regulations, §156. 

I hereby claim foreign priority benefits under Trtle 35, United States Code, §119 of any foreign 
application(s) for patent or inventor's certificate listed below and have also identified below any foreign 
application for patent or inventor's certificate having a filing date before that of the application on which 
priority js claimed: 

Prior Foreign Application®: Priority Claimed 



(Number) (Country) (Day/Month/Year) Yes No 

I hereby claim the benefit under Title 35, United States Code, §120 of any United States 
application® listed below and, insofar as the subject matter of each of the claims of this application is not 
disclosed in the prior United States application in the manner provided by the first paragraph of Trtle 35, 
United States Code, §112, 1 acknowledge the duty to disclose information material to the patentability of 
this application as defined in Title 37, Code of Federal Regulations, §1.56 which occurred between the 
filing date of the prior application and the national or PCT international filing dale of this application: 
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I hereby declare that all statements made herein of my own knowledge are true and that all 
statements made on information and belief are believed to be true; and further that these statements were 
made with the knowledge that willful false statements and the like so made are punishable by fine or 
imprisonment, or both, under Section 1001 of Title 18 of the United States Code and that willful false 
statements may jeopardize the validity of the application or any patent issued thereon. 

POWER OF ATTORNEY: As a named inventor, I hereby appoint the following attorneys and/or 
agents to prosecute this application and transact all business in the Patent and Trademark Office 
connected therewith. 

David H. Judson, Reg. No. 30,467 
Douglas A. Sorensen, Reg. No. 31,570 
Andrea P. Bryant, Reg, No. 28,191, 

Send correspondence to: David H. Judson, Hughes & Luce, LLP., 1717 Main Street, Suite 2800, 
Dallas, Texas 75201 and direct all telephone calls to Mr, Judson at 214/939-5672. 

FULL NAME OF FIRST INVENTOR: Thomas B. Schalk 

INVENTOR'S SIGNATURE: ^ 



DATE: 



RESIDENCE: 



6637 Muirfietd Circle 
Piano, Texas 75093 



County 



CITIZENSHIP: 



USA 



FULL NAME OF SECOND 
INVENTOR: 




RogerS. Zimmerman 



INVENTOR'S SIGNATURE: 



DATE: 



RESIDENCE: 



32 Hastings Street 

Wellesley, Massachusetts 02481 

Norfolk County 



CITIZENSHIP: 



USA 
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For: 



KNOWLEDGE-BASED STRATEGIES APPLIED TO N-BEST LISTS 
IN AUTOMATIC SPEECH RECOGNITION SYSTEMS 



Applicant: 



Schalk et al. 



VERIFIED STATEMENT (DECLARATION) CLAIMING SMALL 
ENTITY 137 CFR 1.9(f) and 1.27(c)) - SMALL BUSINESS CONCERN 

I hereby declare that I am an official of the small business concern 
empowered to act on behalf of the concern identified below: 



I hereby declare that the above identified small business concern qualifies as 
a small business concern as defined in 13 CFR 121 .3-1 B f and reproduced in 37 
CFR 1 .9{d), for purposes of paying reduced fees under section 41(a) and (b) of Title 
35, United States Code, in that the number of employees of the concern, including 
those of its affiliates, does not exceed 500 persons. For purposes of this 
statement, (1) the number of employees of the business concern is the average 
over the previous fiscal year of the concern of the persons employed on a full-time, 
part-time or temporary basis during each of the pay periods of the fiscal year, and 
(2) concerns are affiliates of aach other when either, directly or indirectly, one 
concern controls or has the power to control the other, or a third party or parties 
controls or has the power to control both. 

I hereby declare that rights under contract or law have been conveyed to and 
remain with the small business concern identified above with regard to the 
invention entitled KNOWLEDGE-BASED STRATEGIES APPLIED TO N-BEST LISTS IN 
AUTOMATIC SPEECH RECOGNITION SYSTEMS, described in the attached 
specification. 

I acknowledge the duty to file, in this application or patent, notification of 
any change in status resulting in loss of entitlement to small entity status prior to 
paying, or at the time of paying, the earliest of the issue fee or any maintenance 
fee due after the date on which status as a small entity is no longer appropriate. 
(37 C.F.R. § 1.28(b)). 



NAME OF CONCERN: 



Voice Control Systems, Inc. 



ADDRESS OF CONCERN: 



14140 Midway Road, Suite 100 
Dallas, Texas 75244 
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I hereby declare that all statements made herein of my own knowledge are 
true and that all statements made on information and belief are believed to be true; 
and further willful false statements and the like so mads are punishable by fine or 
imprisonment, or both, under section 1001 of Title 1B of the United States Code, 
and that such willful false statements may jeopardize the validity of the application, 
any patent issuing therein, or any patent to which this verified statement is 
directed. 
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(Name of Person Signing) 
(Title of Person other than Owner) 



14140 Midway Road, Suite 100 

Dallas, Texas 75244 

(Address of Person Signing) 
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